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Abstract 

The neutrinos of long baseline beams travel inside the Earth's crust where the 
density is p ~ 2.8 g cm' 3 . If electron neutrinos participate in the oscillations, 
matter effects will modify the oscillation probabilities with respect to the vacuum 
case. Depending on the sign of Am 2 an MSW resonance will exist for neutrinos 
or anti-neutrinos with energy E r v es ~ 4.7 • |Am 2 |/(10 -3 eV 2 ) GeV. For Am 2 in 
the interval indicated by the Super-Kamiokande experiment this energy range is 
important for the proposed long baseline experiments. 

For positive Am 2 the most important effects of matter are a 9% (25%) enhance- 
ment of the transition probability P(vn — > v e ) for the KEK to Kamioka (Fermilab to 
Minos and CERN to Gran Sasso) beam(s) in the energy region where the probability 
has its first maximum, and an approximately equal suppression of P^p^ — > v e ). For 
negative Am 2 the effects for neutrinos and anti-neutrinos are interchanged. Produc- 
ing beams of neutrinos and antineutrinos and measuring the oscillation probabilities 
for both the — ► v e and — > V e transitions can solve the sign ambiguity in the 
determination of Am 2 . 



1 Introduction 

The data on atmospheric neutrinos collected by Super-Kamiokande [[I], |2j and other de- 
tectors (Kamiokande, 1MB, Soudan and MACRO [0, [|, [5|, ||) give good evidence for 
the existence of neutrino oscillations with |Am 2 | ~ 10~ 3 -10~ 2 eV 2 . The experimen- 
tal results show that the flux of muon (anti-)neutrinos is suppressed with respect to 
the standard-model prediction while the flux of electron (anti-)neutrinos is compatible 
with the no-oscillation prediction taking into account measurement errors and systematic 
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uncertainties. This suggests that the dominant effect of the oscillations is — > v T tran- 
sitions, in good agreement with the results of the Chooz experiment fl(| that put strong 
limits on possible v e —>■ v x transitions in the relevant region of L/E v . 

It is possible that the observed disappearance of the muon (anti-)neutrinos is due to 
oscillation into a light sterile state J?]], and also other forms of 'new physics' beyond the 
standard model have been proposed as explanations of the atmospheric neutrino data 
(see H for a critical discussion). In this work we will however only consider standard 
oscillations between three neutrino flavors. This is the simplest extension of the standard 
model that can describe the data, and can do it very successfully § . In this framework 
one expects transitions between all flavors iy^ <-> u T , «-> i/ e , v e «-> u T ), and a very 
important goal of future experiments will be the measurement of (or the setting of more 
stringent limits on) the transitions involving electron neutrinos. 

Future experiments measuring the disappearance of reactor neutrinos with longer path- 
lengths (in particular the Kamland detector 0) will be able to study v e — > v x transitions 
down to lower values of Am 2 , however, because of systematic uncertainties [ lOfl , it will 



be difficult to extend the sensitivity of reactor experiments to values of the mixing much 
lower that those already excluded by Chooz. Higher statistics measurement of the atmo- 
spheric neutrino fluxes have a good potential to search for the flavor transitions of the z/ e 's 



Tl|, O, 13] searching for up-down asymmetries in the e-like events. Also experiments [14 



using long baseline (LBL) neutrino beams such as the KEK to Kamioka [15|, Fermilab 
to Minos |16[ and CERN to Gran Sasso []T7| projects have a very interesting potential for 
measuring — > v e transitions. The LBL beams are mostly composed of v^s with a small 
v e contamination below the 1% level, and a detector with good electron identification 
capability, collecting a sufficiently large sample of events, can be sensitive to — > v e 
transitions even for values of the mixing well below the Chooz limit. 

Because of the sphericity of the earth, the neutrinos of LBL beams travel a few kilo- 
meters below the Earth's surface, in a medium that can be considered as approximately 
homogeneous with an electron density n e ~ 1.69 • 10 24 cm -3 (corresponding to a den- 
sity p ~ 2.8 g cm -3 and an electron fraction Y e = n e /(n p + n n ) = 0.5). If the electron 
neutrinos participate in the oscillations, the presence of matter modifies the oscillation 
probabilities. In this paper we will discuss in some detail the matter effects and their 
observable consequences. 

The work is organized as follows: in the next section we introduce the theoretical 
framework used in this analysis (a single relevant Am 2 ); in section 3 we discuss the 
existing limits on the three independent parameters present in this framework (Am 2 and 
two mixing parameters); in section 4 we compute the neutrino effective squared masses and 
mixing matrix in matter; in section 5 we discuss the propagation of neutrinos in matter 
with a constant electron density (a good approximation for LBL beams); in section 6 
we finally discuss the oscillation probabilities for LBL beams. A summary is given in 
section 7. 
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2 One mass scale approximation 



In the general case of mixing between three neutrino flavors, one has to consider three 
masses uii, m 2 and m 3 , and a (3 x 3) unitary mixing matrix U that relates the flavor 
{|^ e ), |z/ M ), \u T )} and mass {|^i), l^), l^)} eigenstates: 

Wa) = U aj \Uj) (1) 

Without lack of generality we can label the mass eigenstates so that: 

I 2 2| ^ I 2 21 /o\ 

\m 3 — m 2 \ > |m 2 — m-J (2) 

and 

m l — m 2 — m 3 ° r m l — m 2 — m 3 (3) 

With this choice is the 'most isolated' state separated by the largest mass difference 
gap from the neutrino closest in mass and is therefore the heaviest (or lightest) neutrino, 
correspondingly \vi) is the lightest (or heaviest) neutrino state. 

In this work we will make the approximation that a single mass scale is important for 
the experiments we are considering. More explicitely we will assume that: 

1. the squared mass differences Amf 2 and Am| 3 are of different order of magnitude: 

|2 2|^|2 2| f a\ 

l m 3 ~ m 2\ ^ \ m 2 ~ m l\i (4) 



2. the mass difference \rn^ — m\\ is too small to give observable effects in the measure- 
ments considered. 

The study of oscillations under these assumptions has been performed by several authors 
in the past [|19|, ^D, |2T| . Formally it corresponds to the study of neutrino oscillation under 



the hypothesis that two neutrinos are degenerate in mass {vri2 — mi) ■ The use of the one 
mass scale approximation is motivated by three type of considerations: 

1. Neutrino oscillations in this approximation can still produce transitions between all 
flavors, and therefore this is a much more general framework than the special case 
of two-flavor mixing. The description of oscillations remains however much simpler 
than in the general case, and therefore the one mass scale approximation represents 
a natural 'minimum model' to study oscillations. 

2. In most theoretical models it is natural to expect that the squared mass differences 
are organized in a hierarchical form \m\ — 7?t|| 3> \m\ — m\\. 

3. Finally and most important, there are strong experimental indications, coming from 
experiments on solar neutrinos J22 that the mass difference Am^ controls the 



observed suppression of the solar neutrino fluxes, and is of order ~ 10~ 5 eV 2 (for 
the MSW solutions) or ~ 10 -11 eV 2 (for the just-so solution). These low values of 
Am 2 2 are unobservable in long-baseline experiments and can result in at most small 
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effects p3| , 33| for low energy atmospheric neutrinos. Therefore if oscillations are 
the solution for both the atmospheric and the solar neutrino problems the one mass 
scale approximation is a completely adequate framework to describe long-baseline 
experiments, and is a reasonable model for the study of atmospheric neutrinos. 



When two neutrinos are degenerate in mass the mixing matrix U is not completely 
defined. In fact the j-th column of the matrix (U e j, U^j, U T j) describes the flavor content 
of the mass eigenstate \vj), and the two states with the same mass and \u2) can be 
chosen in an infinite number of ways. The degeneracy is broken when neutrinos propagate 
in matter, because the v e and (z/ T ) have different potentials (see next section), and in 
the subspace of the neutrino states orthogonal to \v%) the states with and without a \u e ) 
component have different effective masses. It is therefore convenient to choose one of the 
eigenstates, for example the \ui) state, as orthogonal to the flavor state \u e ), this also 
determines l^) (modulo some phase convention) as the normalized state orthogonal to 
\ui) and | u 3 ). With these conventions the mixing matrix U can be parametrized as: 



U 



cos 9 sin 9 

cos ip — sin 9 sin (p cos 9 sin ip 
— sin ip — sin 9 cos ip cos 9 cos ip 



(5) 



with the two mixing angles 9 and ip defined in the interval [0, 7r/2]. The definition of the 
mixing matrix in equation (|5|) contains some arbitrary sign conventions (only the absolute 
values \U a j \ of the elements are uniquely defined), the important point is that it is possible 
to define all elements as real and all CP or T violating effects vanish in the one mass 
scale approximation. In the following we will also use the notation 



Pa3 = 1(^1^3) 



\u, 



a ?, I 



(6) 



where p a % is the probability that the neutrino lz/3) has the flavor \v a ). These probabilities 
obviously satisfy the relation: 

Pe3 + £7*3 + Pt3 = 1 (7) 

In the one mass scale approximation all oscillation effects (also in the presence of 



matter) can be expressed as a function of Am z 



m{ and of the three probabilities 



Pe3, P/j,3 and p T 3. The transition probabilities in vacuum can be written as: 

~Am 2 L~ 



pvac 



t {L/E u \ 

and the survival probabilities as: 

PZ° (L/E v ) = l-ip ea (l 



4p«3P/33 sin 



4£„ 



Pa3) Sill 



Am 2 L 



For fixed E v all probabilities oscillate with a single oscillation length \q(E v 

4ttE u n lnn E u (GeV) 



\(E V 



I Am 2 



2.470 



Am 2 (eV 2 ) 



km 



(8) 



(9) 



(10) 



that grows linearly with increasing E v 
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3 Experimental results 



The available information on the oscillation probabilities in the region of interest are sum- 
marized in fig. |T| where we show the allowed regions obtained by the Super-Kamiokande 



(shaded area) and the Chooz experiment. The curves are taken from |2| and [10 where 
they were obtained in the framework of of a two flavor oscillation analysis (we have only 
relabelled the x-axis as A ee for Chooz limit and A^ T for the SK result). 

We want to translate these results into constraints on p e3 , p^ 3 and p r3 . In the Chooz 
analysis the electron neutrino survival probability is described with the form P v ^ Ve = 
1 — A ee sin 2 [Am 2 /4:E U ], and an allowed region is determined for the parameters Am 2 
and A ee . The form of the probability is identical to the one obtained in the framework 
of the one mass scale approximation, and using A ee = 4p e3 (1 — p e3 ) for any Am 2 the 
allowed interval in A ee can be translated in an allowed interval on p e3 (composed of two 
disconnected sub-intervals) . 

The SK analysis is valid only in the framework of two flavor (z/ M «-> u T ) oscillations, 
since it is assumed that the v e and V e fluxes are not modified. In the framework of the 
one mass scale approximation, this corresponds to the limit p e3 = 0. In this limit, using 
A^ T = 4 p M 3 p r3 , the allowed region in A^ T can be translated in allowed intervals for p^ 3 
and p r3 . Since the Chooz results tell us that p e3 is small we can consider these intervals 
as a reasonable approximation. Taking into account the possibility of a non vanishing p e3 
the allowed interval for p^ 3 (or p r3 is slightly enlarged (for a detailed discussion see (Tl| ) . 

The results are shown in fig. ^. As an illustration of how to read the figure we can 
consider the value | Am 2 | = 3 x 10 -3 eV 2 . For this value of the squared mass difference the 
Chooz upper limit (A ee < 0.13) tells us that the state \u 3 ) is either a quasi pure electron 
neutrino state or contains only a small overlap with \u e ): 

Pes < 0.033 or p e3 > 0.966 (11) 

The SK results tell us that there are oscillations between and v T and the amplitude of 
the oscillations is close to unity (A^ T > 0.86). This can be translated as: 

0.32 </V3,Pr3< 0.68 (12) 

Of course the three probabilities are constrained to satisfy p e3 + p^ 3 + p r3 = 1, therefore 
only the 'small e-flavor' interpretation of the Chooz data remains acceptable, and a large 
(small) p^ 3 implies a small (large) p r3 . We note that the best fit of Super-Kamiokande 



sin 



26 ^ T = 1) corresponds to p^ 3 = p r3 = | and p e3 = 



A central goal of future experiments will be to measure more accurately the flavor 
content of the neutrino state lz/3), and in particular to measure (or put a more stringent 
limit) to the overlap between the lz/3) and |z/ e ) states. 

We note that, in analogy with the charged lepton and quark masses, it is 'natural' 
to expect that the state \v 3 ), that is experimentally determined to be a combination 
with approximately equal weights of muon and tau neutrinos and a small (or vanishing) 
\u e ) component, is the heaviest neutrino. This however is an assumption that could be 
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false, and that should be experimentally verified. No oscillation experiment in vacuum 
can solve this ambiguity, since the mass dependent oscillating term sin 2 [Am 2 L/(AE V )] is 
invariant for a change of sign in Am 2 . If however neutrinos are propagating in matter, 
measurements of flavor transitions can determine the sign of Am 2 , that is determine if 
the state ^3) is the heaviest or lightest neutrino. 



4 Matter effects 

When neutrinos propagate in matter the interactions with the medium results in a a 
flavor dependent effective potential V a ilSfl . The difference in effective potential between 
electron neutrinos and (tau) muon (anti)-neutrinos is: 



V = V e - V u = V e - V T = ±V2 G F n e 



(13) 



where n e is the electron density of the medium, Gf is the Fermi constant and the plus 
(minus) sign applies to neutrinos (anti-neutrinos). Using the relation 



p 2 + m? +V 



m 

P+ 2t + V 



(14) 



the potential can be considered as a contribution 5m 2 ; (z/ e ) 

3ig 

in matter U m can calculated solving the equation: 



2E V V to the effective squared 



mass of z/ e 's. The neutrino effective squared mass eigenvalues M 2 and the mixing matrix 



U 



m\ 







ml 






ml 



U 



T 



2VE U 





u„ 







Ml 






Ml 








u 



(15) 



The columns of the matrix U m give the flavor components of a new set of 'propagation 
eigenvectors' {|z/i m ), |z/3 m ), |z/3 m )} with well defined effective mass in matter. 



The matrix diagonalization problem of equation (15) is very simple under the assump- 



tion m 2 = m| and has been discussed before by several authors [TJ3|, O, [TJ|. The neutrino 
state has been chosen (see previous section) as having no electron flavor component, 
and therefore is decoupled from all matter effects, and the problem is equivalent to the 
well known case of two flavor mixing. The effective mass eigenvalues and the mixing 
matrix can be written as a function of the adimensional quantity x: 



x 



2VE U _ ± 2^/2 G F n e E v 
Am 2 Am 2 



±0.076 



pjgcm- 3 ) E u (GeV) 
Am 2 (10" 3 eV 2 ) 



(16) 



the plus (minus) sign applies to z/'s (J7's t 
assumed an electron fraction Y e = 0.5. 

The effective squared mass eigenvalues are: 



For the numerical estimate in (TTBT) we have also 



M 2 



m 2 
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M? = m? + Am 2 - 
2 2 

Mf = m? + Am 2 - 
i 2 



1 +x - ^ sin 2 20 + (x-cos20) 2 



(17) 



1 + x+ v 4in 2 20 + (x-cos20) 2 



The 



mixing matrix in matter U m has the same form as in the vacuum case: 





cos if 



cos6 m (x) 



sin 6 m [x) 



sin m (x) sin <£> cos8 m (x) sin ip 
— sin if> — sin 6 m (x) cos y? cos 6 m (x) cos ip 



but the angle 6 m is a function of the parameter x: 



sin 6 m (x) 



x - cos 29) + ^sin 2 26 + (x - cos 26) 2 

i 

2 



{[(a; -cos 26) + Jsin 2 20 + ( x — cos 2d) 2 ] 2 + sim 26} 



(19) 



The resulting sin 2 26 m {x) has the well known expression: 



sin 26 m (x) 



sin 2 26 



sin 2 20 + (x-cos20) 2 



(20) 



Note that we have given explicitely the three mass eigenvalues (and not only the 
difference Mf — Mf), and also sin0 m (and not only sin 2 20 m ), because these quantities are 
needed to compute 3-flavor oscillations, as we will discuss in the following. 

An illustration of the effect of matter on the masses and mixing of the neutrinos is 
shown in figures 3, 4, 5 and 6. For these figures we have assumed Am 2 = 3 • 10~ 3 eV 2 , 
a probability p e z = \U e 3\ 2 = sin 2 6 = 0.025 (the specification of and U T 3 is not 
necessary) and that the neutrinos (or anti-neutrinos) propagate in matter of constant 
density p = 2.8 g cm -3 (with electron fraction Y e = 1/2). This last condition is a good 
approximation for neutrinos in LBL beams that travel few kilometers below the Earth's 
surface. In fig. |^ we show the neutrino effective mass eigenvalues M 2 (equation [17]) plotted 
as a function of E v . Fig. ^ is the same but for antineutrinos. In fig. |5| (fig. |(J) we show the 
values of sin 2 m (sin 2 20 m ) (again plotted as a function of E„), the solid (dashed) lines 
refer to z/s (l^'s). 

Note that in matter the propagation eigenvectors \ui m ) = \ui) and \v2m) ar e not 
anymore degenerate, since the effective mass of \ui) can be considered as constant while 
the effective squared mass of the state \v2m) {\^2m) ) increases (decreases) with increasing 
n e E u . 

Inspecting the equations that describe the effective masses and mixing in matter, we 
can notice that there are some special conditions: 

1. The case x = is simply the vacuum case: M 2 (0) = m 2 and 6 m (0) = 6. 

2. The case x — > +oo corresponds to the limit of very large density or very large E v for 
neutrinos (anti-neutrinos) if Am 2 > (Am 2 < 0). In this situation the state |z/ 3m ) 
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becomes a pure \u e ) (0 m — > tt/2) with a very large effective mass Mf — > oo, and 
decouples from the oscillations; the state \ v2m) asymptotically has the effective mass 
Mf — > m 2 + Am 2 cos 2 0. Note that in this condition of very strong matter effects, 
the oscillations involving v e are completely suppressed, but <-> v T oscillations do 
occur with a maximum probability 4 sin 2 ip cos 2 ip larger than the vacuum one by a 
factor (cos 2 #) _1 , and an effective Am 2 smaller than the vacuum value by a factor 
cos 2 0. 

3. The case x — > — oo corresponds to the limit of very large density, or very large 
E u for anti-neutrinos (neutrinos) if Am 2 > (Am 2 < 0). In this situation the 
state \v2m) becomes a pure \v e ) (0 m 0) with effective mass Mf -> -oo; the 
state |z/ 3m ) asymptotically gets the mass M| — > m 2 + Am 2 cos 2 6*. This situation is 
experimentally undistinguishable from the previous case. 

4. Finally we have the special case: 

x = cos 20 = 1 - 2p e3 (21) 

This case can only happen for neutrinos (anti-neutrinos) if Am 2 > (Am 2 < 0). 
It corresponds to the celebrated MSW |L8| resonance. For a fixed electron density, 
the resonance happens at a value of the neutrino energy: 

2V \10- 3 eV 2 J V P J \Y e J K ! 

At the resonance the mixing angle in matter is m = 45°, the probability P[y e ~^ v e) 
oscillates with amplitude sin 2 20 m = 1, while the probabilities P{y e — > v^) and 
P{y e —> v t) oscillate with amplitudes sin 2 ip and cos 2 ^. 



5 Propagation in matter of constant density 

For neutrinos propagating in matter of a constant density the oscillation probabilities can 
be written in general as the superposition of three oscillating terms corresponding to the 
three effective squared mass differences: 



Pu a -*up{L, E v - x)=Yj A ap( X ) Sin2 
j<k 

The amplitudes of the oscillatig terms are given by: 



7T- 



\ jk {x, E v ) 



A TJ m TJ m TJ m TJ m 

4 U aj U /3j U ak U /3k 



The three oscillation lengths are: 

X jk (E u , x) 



4ttE„ 



\M]-M 2 \ 



(23) 



(24) 



(25) 
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and satisfy the relation: 



X\2 



^23 







They can be written explicitely as a function of the parameter x: 

1 



X 12 (E U , x) 
X 13 (E U , x) 
X 23 (E U , x) 



\{E V ) 
X (E U ) 
X {E V ) 



1 + x- \/F(x] 
1 



x + 
-l 



F(x 



Fix) 



(26) 

(27) 
(28) 
(29) 



(30) 



where we have introduced the definition 

F(x) = sin 2 29 + (x - cos 29f 

In fig. [7|we show as an example a plot of the oscillation lengths Xjk calculated as a function 
of the neutrino energy for the same values of the neutrino masses, and mixing and the 
same density of the medium as in the previous figures. Some features are immediately 
visible looking at fig. [7]: 

1. The presence of matter introduces an energy independent length: 



2ir 

A m (n e ) = — = 1.16-10 4 km 



1.69 ■ 10 24 cnr 



2. When x — > (low n e E u ) we have A12 
n e E u ) we have X 12 -> A , Ai 3 , A 23 -> K, 



Am , Al3, A 



13, ^23 



Aq. When x 



(31) 
00 (high 



3. The resonance condition can also be expressed as Xq(E u ) = X m (n e ) cos 2$. At the 
resonance the length A23 has a maximum: A23 = Xq/ sin 29. 

For a fixed value of E v the oscillation probabilities involving v e (or z/ e 's) have a simple 
sinusoidal dependence on L with oscillation length A23. This can be understood immedi- 
ately observing that the eigenstate has no electron flavor component. Formally we 
have that U™ = 0, and therefore from ( f24D follows that: 

(32) 



.4 



12 



A 



13 

ep, 



A 



12 



.4 



13 







therefore the transitions v P 



and v e 



v T develop with a single oscillation length 



A23. The probability Piy^ <-> z/ T ), and therefore also P{y^ 



have a more complex 



functional form because all the three oscillating terms in (|23|) are non vanishing. The non 



vanishing probability amplitudes can be written as: 



A 23, 



a23 

A 12 



A 13 



x) 
x) 

jl,T\ X ) 



a/3 

= 4 sin 2 9 m cos 2 9 m sin 2 
= 4 sin 2 9 m cos 2 9 m cos 2 <p 
—4 sin 2 9 m cos 2 9 m sin 2 ip cos 2 ip 
= 4 sin 2 9 m sin 2 (p cos 2 ip 
= 4 cos 2 9 m sin 2 ip cos 2 <p 



sin 2 29 m sin 2 tp 
sin 2 2^ m cos 2 <p 

• 2 no -2 2 

— sin 29 m sin 9? cos (p 



(33) 
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5.1 Small L regime 

It is interesting to study the oscillation probabilities in matter in the limit of short neutrino 
pathlength L, or more precisely in the limit where L/Xjk <C 1 for all three oscillation 
lengths. The condition of 'small V is going to be valid in the range E v > 10 GeV for the 
long-baseline beams such as the Fermilab to Minos and CERN to Gran Sasso projects 
(L ~ 730 km) if |Am 2 | is in the range indicated by the atmospheric neutrino data. 

It the limit of small L the oscillation probabilities in matter are undistinguishable 
from the vacuum case, even if the effective masses and mixing in matter are very different 
from the vacuum case. This can be demonstrated observing that when L/Xjk is small the 
oscillating terms in can be approximated as: 

T \ 2 

(34) 





and the oscillation probabilities take the form: 

^ = (^) 2 Et^§^ (35) 

j<k A jk\ rj vi X ) 

There are now two relations between the amplitudes Aig and the oscillations lengths Xjk 
that result in a cancellation of the x dependence of the expression fl35l). The first relation 
is present also in the case of two flavor mixing: 



sin 26 m {x) /sin 2 20 \ \jF(x)\ sin 26 



X 2 23 (E u ,x) \ F(x) J \ X Q (E U ) / Xl{E v 



(36) 



where we have used equations (^) and fl2"0|) in the first equality. Similarly it is easy to 
prove that: 

sin 2 6L cos 2 6L cos 2 6 , , 

~l^ + ^^ = ^- (37) 

A 12 A 13 A 

From equation (|33|) we can see that v4 2 ^ oc sin 2 2$ m , oc sin 2 6 m and oc cos 2 9 m , 
and using the relations (ffl) and (|37D in (|35D one can see that for L/Xjk small the matter 
effects have no experimentally detectable effects. 

To illustrate this point in fig. | and fig. |9] we show the transition probability P{v lx — > v e ) 
and Pivy, — > v T ) plotted as a function of the distance L for neutrinos and anti-neutrinos 
of a fixed energy E v = 13.0 GeV traveling in vacuum or in matter of constant matter 
with density p = 2.8 g cm -3 . For this figures we have assumed again for illustration 
Am 2 = 3 • 10- 3 eV 2 , p e3 = sin 2 9 = 0.025 and = p r3 . The energy E v = 13.0 GeV 
is close to the neutrino resonance energy (13.4 GeV) for the density and mass matrix 
considered. Note that the maximum pathlength considered in these plots L = 5 • 10 4 km 
is approximately four times longer than the Earth's diameter and therefore in most of the 
range of pathlengths considered the condition considered cannot be realized in practice. 
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Looking at the figure it is immediately evident that the oscillation properties of the 
neutrinos in matter and in vacuum are dramatically different, however experiments with 
a baseline of L = 250 or 730 km will not measure significant departures from the vacuum 
oscillation probabilities. 



6 Oscillation probabilities for LBL experiments 

In a LBL experiment the neutrino pathlength L and the electron density n e (y) along the 
trajectory (y G [0, L] is a coordinate along the neutrino path) can be considered as fixed, 
therefore what are experimentally accessible, are the oscillation probabilities P Ua ^ Uf3 (E l ,) 
as function of the neutrino energy. In the following we will also use the approximation to 
consider n e as constant. 

If L and n e are fixed, in the one mass scale approximation for a long baseline experi- 
ment one has to consider two characteristic neutrino energies: 



and 



I Am 2 



|Am 2 | L 
2~k 



I Am 2 



0.59 I Am 2 . 



(— 

V730 km 



GeV 



4.70 |Am 2 3 | 



1.69 - 10 24 cm" 3 ' 



GeV 



n,, 



(38) 



(39) 



2V 2 v / 2G/n^ 

where Am 2 3 is the value of the neutrino squared mass difference in units of 10~ 3 eV 2 . 
The energy e Q corresponds to the highest neutrino energy for which the vacuum transition 
probabilities have a maximum (or in different words it is the energy such that Xq(bq) = 
2L). The energy e m is the resonance energy in the limit of small p e3 : 



El 



e m COS26 1 = e m (1 - 2p e3 ), 



(40) 



the parameter x that controls the importance of the matter effect is given by \x\ = 
E v /e m , and therefore for E v <C e m the presence of matters has a negligible effect. Both 
characteristic energies are proportional to Am 2 , and the ratio eo/^m is independent from 
the neutrino mass matrix. For the Fermilab and CERN projects one has e /£ m — 0.125, 
for the KEK to Kamioka (K2K) project e^je m = 0.042. The smalless of this ratio ensures 
that the matter effects are only a correction to the oscillation probabilities. 

In fig. [lO] and fig. [TT] as an illustration we show the transition probabilities Piv^ — > v T ) 
and P(iSfj, — > v e ) plotted as a function of E v for neutrinos that have traveled the distance 
L = 730 km in vacuum (solid line) or in matter with constant density p = 2.8 g cm -3 
(dashed line for neutrinos, dot-dashed line for antineutrinos) . In these examples we have 



used Am' 



3 • 10-' eV 2 



Pe3 



sin 9 = 0.025, and p^ 3 = p r3 = 0.4875. The vacuum 



transition probabilities have a simple sinusoidal form: 



p 



A, 



,p sin 



7T £o_ 
2 E u 



(41) 
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with maxima at E v = Eq/u (n is a positive integer) where the probability has a value 
A a /3 = 4:Pa3P/33- The presence of matter (for p e3 ^ 0) results in some deviations of the 
oscillation probabilities from the form ([11]). The effect, for Am 2 > 0, is an enhancement 
(suppression) of the — > v e (V^ — > V e ) transition and correspondingly a suppression 
(enhancement) of the —>■ v T (z/ /t — > V T ) transition. The absolute size of AP = P ma t—Pvac 
is similar in both — > v e and — > v T oscillation, but in this second case the effect is 
much more difficult to detect, and less important to consider, because it represents a small 
correction to a probability of order unity. 



Looking at fig. [10] and fig. |TT] it is possible to observe that the matter effects are small 
both for large and small E v and most important in an intermediate energy. This is true in 
general and can be easily understood qualitatively. We can in fact consider three energy 
regions: 

1. Large E u (or more precisely E u ^> Eq). In this region the oscillation length is much 
longer than the pathlength L {L/\q{E v ) = e /2E u ), and therefore P mat ~ P vac for 
the reasons described in section 5.1. 

For the existing long baseline projects, neutrinos with energy close to the MSW 
resonance (E ) belong to this region since E m /£o is large, therefore the presence 

of matter does not result in large visible effects for neutrinos at the resonance, even 
if the effective masses and mixing are very different from the vacuum values. 

2. Small E u (or more precisely E u <C e m ). In this region the effective masses and 
mixing of neutrinos in matter are close to the vacuum values (\x\ = E v /e m <C 1) 
and again one has P ma t — Pvac- 

3. Intermediate E u . In this region the neutrino energy is not much smaller that e m 
and it is not much larger than Eq. The first condition is needed to have significant 
modifications of on the effective masses and mixing, the second one to have a suf- 
ficiently large L/Xq. For the projected long baseline beams the two conditions can 
never be fully satisfied at the same time (because e Q /e m < 0.125 and this is why the 
matter effects never produce very important effects. When the neutrino energy is 
E u ~ e the conditions required to have significant matter effects are best satisfied, 
and this is where the matter effects manifest themselves most clearly. 



Figures [T^ and |TT| have been calculated for a specific value of Am 2 , but they describe 
oscillations also for an arbitrary value of Am 2 . In fact the oscillation probabilities, for 
any distribution of matter along the neutrino path, are a function of E u /Am 2 , and the 
figures be considered as valid for different values of Am 2 simply rescaling the neutrino 
energy (for negative Am 2 the probabilities for neutrinos and antineutrinos have to be 
interchanged). It remains to discuss matter effects for different neutrino mixings. This is 



illustrated in fig. [12| that describes the transition probability Piy^ u e ) for the Fermilab 



to Soudan and CERN to Gran Sasso projects for an arbitrary value of Am 2 and for all 
allowed possible mixings. In the figure we plot the probability Piv^ — > v e ) divided by 
^Pe3P[i3 (that is the amplitude of the oscillation probability in vacuum) as a function of 
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E u /e for neutrinos that have traveled a distance L = 730 km in vacuum (solid line) or 
in matter with constant density p = 2.8 g cm~ 3 (dashed line for neutrinos, dot-dashed 
line for antineutrinos). All curves are valid for all values of p^, and all positive values 
of Am 2 (for Am 2 < the curve of v and V have to be interchanged). The probability 
has beeen calculated for three values of the mixing p e3 = sin 2 9 = 0.03, 0.02 and 0.01. 
The three curves for vacuum oscillations, are identical. The curves calculated taking into 
account the matter effects are very close to each other, and for p e3 — > they tend to an 
asymptotic constant form. This can be simply understood observing that the v\i — > u e 
(or I7 M — > VI) probability can be written as: 



.uSEu) = -j^- sm 



Am 2 , Fix) L 



4£„ 



(42) 



where x = ±E u /e m (the plus (minus) sign is for i/s (z^'s)), and F(x) = sin 2 29 + (x — 
cos29) 2 . When p e3 = sin 2 9 is small (as indicated by the combined analysis of Chooz and 
Super-Kamiokande), F(x) can be well approximated as F(x) ~ (1 — x) 2 , and the ratio 
P( v fi U e)/Pe3 becomes independent from p e3 : 



sm 



7T E (1 - X) 

2 £■„ 



(43) 



The main effect of matter on Piy^ — > z/ e ) is an enhancement or suppression of the prob- 
ability at the first maximum P*( V y. 



P*,-, ~ P* 

u(u) vac 



(44) 



where P^ ac = 4:p^ 3 p e3 . The minus (plus) sign refers to z/'s (F's) for positive Am 2 or 
viceversa for negative Am 2 . The matter effects result also in a displacement of the energy 
E*^ of the first maximum. Numerically (assuming Am 2 > 0) for the Fermilab and CERN 
project (L = 730 km) one has: 



Ft ~ P; ac • 1.26, 

E* v ~ e ■ 0.92, 
For the K2K project (L = 250 km) one finds: 



Pv - Plac ■ 0-76 
E; ~ e • 1-07 



(45) 
(46) 



p* . I 09 

J vac -L-Uf, 



5 • 0.98, 



p_f p* 



0.92 



~ e • 1.02 



(47) 
(48) 
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7 Discussion and conclusions 



The effect of matter on neutrino oscillations in the projected long-baseline neutrino beams 
is small but detectable, especially for the longer pathlength and higher intensity Fermilab 
to Soudan and CERN to Gran Sasso beams. 

In these experiments, assuming Am 2 is positive, the effect of matter will manifest itself 
as an approximately 25% enhancement (suppression) of the — > v e (V^ — > V e ) probability 
in the crucial region of the first maximum. The energy E v where the probability has the 
first maximum will also be lower (higher) than in vacuum by approximately 7%. In 
the shorter pathlength K2K project the matter effects will result in smaller effects: a 9% 
enhancement of the probability and a 2% displacement of the energy of the first maximum. 

It is interesting to observe that the detectable effects of matter on the oscillations are 
most important not when E v is close to the MSW resonance, but at lower energy when the 
modifications induced by matter on the neutrino effective masses and mixing are smaller, 
but when the oscillations can develop because of a shorter oscillation length (comparable 
with L) and the modifications of the oscillation parameters can produce visible effects. 

If Am 2 , in contrast with the expectations, is negative, then the effects of matter on 
neutrinos and anti-neutrinos are reversed. This is perhaps the most interesting effect, be- 
cause it provides a method to measure the sign of Am 2 resolving the existing ambiguity. 
In order to do this one needs to produce (during different periods of data taking) beams 
of both neutrinos and anti-neutrinos inverting the polarity of the focusing system down- 
stream of the target region, and to detect both — > v e and — * V e transitions, measur- 
ing the oscillation probabilities for energies E u ~ e . The ratio Piy^ — > v e )/P(y^ — > V e ) 
can have only two possible values (approximately 1.64 or 0.61 for the Fermilab and CERN 
projects, 1.18 or 0.85 for the K2K project) depending on the sign (positive or negative) 
of Am 2 . 
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Figure 1: Limits on — > v T and v e — > z/ e transitions obtained by the Super-Kamiokande 
and Chooz experiments. 



17 




0.25 0.5 0.75 1 

Pe3> PjLt3' Pt3 



Figure 2: Limits on the flavor content p a3 = |(f Q |^3)| 2 of the neutrino state |z/ 3 ) obtained 
from the results of the SK and Chooz experiment. The SK results give an allowed interval 
for p^ 3 and p r3 ; the Chooz results give an allowed interval for p e3 . 
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Figure 3: Effective mass eigenvalues for neutrinos propagating in matter with density 
p = 2.8 g cm" 3 (and electron fraction Y e = n e /(n p + n n ) = 1/2) plotted as a function of 
E v . The neutrino masses are m\ = m\, and m\ = m\ + 3 ■ 10~ 3 eV 2 . The neutrino state 
\u 3 ) as a probability p e3 = sin 2 9 = 0.025 of having electron flavor. 
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Figure 4: Effective mass eigenvalues of anti-neutrinos, propagating in matter with density 
p = 2.8 g cm -3 . The neutrino masses and mixing are the same as in fig. 1. 
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Figure 6: Value of sin 2 29 m plotted as a function of E u for neutrinos (solid line) and 
antineutrinos (dashed line) propagating in matter with density p = 2.8 g cm" 3 . In 
vacuum p e3 = sin 2 9 = 0.025 (sin 2 29 = 0.0975). The neutrino masses are m\ = m 2 , 
ml = m\ + 3 • 10" 3 eV 2 . 
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Figure 7: Oscillation lengths = 4kE v /\M? — M|| plotted as a function of E u for 
neutrinos traveling in matter of density p = 2.8 g cm' 3 . The neutrino masses are mi = m 2 , 
ml = m\ + 3 • 1CT 3 eV 2 , the electron flavor content of the state |z/ 3 ) is p e3 = sin 2 9 = 0.025. 
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Figure 8: Transition probability Piy^ — > v e ), plotted as a function of the distance L for 
neutrinos with a fixed energy E v = 13.0 GeV traveling in vacuum (solid line) or in matter 
of constant matter with density p = 2.8 g cm" 3 (thick solid line for neutrinos, dashed line 
for antineutrinos). The neutrino masses are mi = rri2, m\ = m 2 + 3-10 -3 eV 2 . The mixing 
matrix in vacuum is determined by p e s = sin 2 9 = 0.025 and p^ 3 = p r3 (sin 2 if = 0.5). 
E u = 13.0 GeV is close to the resonance energy for the density and parameter values 
considered. 
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Figure 9: Transition probability Piy^ — > v T ), plotted as a function of the distance L 
for neutrinos with a fixed energy E u = 13.0 GeV traveling in vacuum (thin solid line) or 
in matter of constant matter with density p = 2.8 g cm -3 (thick solid line for neutrinos, 
dashed line for antineutrinos). The neutrino masses are mi = ra 2 , m\ = m 2 + 3- 1CT 3 eV 2 . 
The mixing matrix in vacuum is determined by p e3 = sin 2 6 = 0.025 and = p r3 
(sin 2 if = 0.5). E v = 13.0 GeV is close to the resonance energy for the density and 
parameter values considered. 
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Figure 10: Transition probability Piv^ — > u T ) plotted as a function of E u for neutrinos 
that have traveled a distance L = 730 km in vacuum (solid line) or in matter with constant 
density p = 2.8 g cm -3 (dashed line for neutrinos, dot-dashed line for antineutrinos). The 
neutrino masses are mi = m 2 , m\ = ml + 3 ■ 10~ 3 eV 2 . The mixing matrix in vacuum is 
determined by p e s = sin 2 6 = 0.025 and = p r3 (sin 2 ip = 0.5). 
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Figure 11: Transition probability -P(V^ — > v e ), plotted as a function of E v for neutrinos 
that have traveled a distance L = 730 km in vacuum (solid line) or in matter with constant 
density p = 2.8 g cm -3 (dashed line for neutrinos, dot-dashed line for antineutrinos). The 
neutrino masses are mi = m 2 , ml = ml + 3 ■ 1CT 3 eV 2 . The mixing matrix in vacuum is 
determined by p e s = sin 2 9 = 0.025 and = p r3 (sin 2 ip = 0.5). 
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Figure 12: Transition probabilities Piv^ — > u e ), plotted as a function of E u /eq for 
neutrinos that have traveled a distance L = 730 km in vacuum (solid line) or in matter 
with constant density p = 2.8 g cm -3 (dashed line for neutrinos, dot-dashed line for 
antineutrinos). All curves are valid for all values of and all positive values of Am 2 , 
for negative Am 2 the neutrino and anti-neutrino curves have to be interchanged. The 
probability has beeen calculated for p e3 = sin 2 9 = 0.03, 0.02 and 0.01. The three curves 
for vacuum oscillations, are identical, the curves calculated taking into account the matter 
effects are very close to each other. 
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